Target Audience: MEL specialists, program managers, evaluation professionals, and senior development practitioners leading evidence-based programs
Prerequisites: MEL 101 Workshop 1 or equivalent experience with basic MEL concepts
Materials Needed: Sample evaluation reports, data analysis tools, case study materials
By the end of this workshop, participants will be able to:
Scenario: Government launches conditional cash transfer program for poor families. After 2 years, school enrollment increases by 15% in program areas. Politicians claim success, but critics argue enrollment was already rising due to new schools being built.
The Question: How much of the enrollment increase can we attribute to the cash transfer program versus other factors?
The Challenge: This is the fundamental attribution problem in evaluation—separating program effects from everything else happening simultaneously.
Gold Standard: Random assignment to treatment/control
When to use: New interventions, policy pilots, sufficient sample size
Limitations: Ethical concerns, implementation challenges, external validity
Natural Experiments: Exploit policy variation or eligibility rules
When to use: Program already implemented, ethical RCT concerns
Limitations: Stronger assumptions, potential confounders
Mixed Methods: Compare changes over time with similar areas
When to use: Limited data, smaller programs
Limitations: Selection bias, unobserved differences
Process Focus: Test how and why programs work
When to use: Complex interventions, innovation, learning focus
Limitations: Less definitive attribution
Challenge: Choose evaluation methods for these scenarios:
| Scenario | Key Question | Constraints | Your Method Choice |
|---|---|---|---|
| New microfinance product pilot | Does it increase business income? | 6 months, 500 clients, ethical to randomize | _____________ |
| National nutrition program (5 years old) | Impact on child stunting rates | Already implemented, no control possible | _____________ |
| Community-led development innovation | How does it work? Why successful? | Complex, context-dependent, learning focus | _____________ |
Discuss: What would be your second-choice method for each scenario? Why?
Traditional evaluation often fails because:
Participatory evaluation addresses these by:
Process: Stakeholders collect and analyze stories of change
Best for: Complex programs, unexpected outcomes, empowerment
Process: Verify and interpret outcomes with stakeholders
Best for: Innovation, complex change, unclear outcomes
Process: Communities assess service quality
Best for: Service delivery, social accountability
Process: Communities create videos about change
Best for: Marginalized voices, advocacy, storytelling
Context: NGO implements community water systems across 50 villages in Rajasthan. Traditional evaluation would measure functionality rates and usage statistics.
Participatory Approach Used:
Key Insights Uncovered:
Result: Program redesigned based on community priorities, sustainability improved significantly
Scenario: You're evaluating a youth livelihoods program in urban slums. Young participants have diverse backgrounds, limited literacy, but high mobile phone usage.
Your Task: Design a participatory evaluation approach
Which participatory methods would you use? Why?
_____________________________
How would you train youth to be evaluators?
_____________________________
How could mobile phones enhance the evaluation?
_____________________________
How would you ensure findings lead to action?
_____________________________
Traditional Approach: Plan → Implement → Evaluate → Report
Adaptive Approach: Plan → Implement → Monitor → Learn → Adapt → Repeat
Continuous monitoring and rapid feedback
Regular reflection and analysis
Extract insights and implications
Translate learning into action
Adjust strategy and implementation
The Data-to-Decisions Pipeline:
| Stage | Activities | Tools/Methods | Key Questions |
|---|---|---|---|
| Data Collection | Rapid, regular data gathering | Mobile surveys, dashboards, sensors | What data do we need when? |
| Data Analysis | Pattern identification, trend analysis | Simple analytics, visualization | What is the data telling us? |
| Interpretation | Context, causation, implications | Team reflection, stakeholder input | What does this mean for our program? |
| Decision-Making | Choose actions based on evidence | Decision frameworks, scenarios | What should we do differently? |
| Implementation | Execute changes, monitor effects | Change management, tracking | How do we implement and track changes? |
Program: Adolescent girls' program in Bihar - life skills, health education, livelihoods training
Initial Challenge: Low attendance in life skills sessions
Adaptive Response:
Result: Final attendance 85%, with mothers becoming program advocates
Challenge: Design an adaptive MEL system for a complex sanitation behavior change program
Context:
What would you track monthly to catch problems early?
How would you ensure regular reflection and adaptation?
What systems/tools would support rapid data use?
Who would make adaptation decisions? How often?
Stakeholder buy-in, capacity assessment, system design
Test systems, train staff, refine processes
Full implementation, quality assurance, integration
Embed in operations, sustain beyond project
Technology-Enabled MEL:
Methodological Innovations:
Participatory and Decolonizing Approaches:
The evolution of MEL is moving from:
MEL practitioners must evolve to become facilitators of learning, change agents, and systems thinkers.
Impact Evaluation and Methods:
Participatory and Learning-Oriented Evaluation:
Systems and Complexity:
Technology and Innovation:
Professional Development:
Indian MEL Community:
Next Steps in ImpactMojo:
This handout is part of the ImpactMojo 101 Knowledge Series
Licensed under CC BY-NC-ND 4.0 • Free to use with attribution • www.impactmojo.in
For advanced evaluation templates, adaptive management tools, and participatory evaluation guides, visit the ImpactMojo platform.
